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REMARKS 

Claims 1, 3-9, and 1 1-18 are pending in this application. By this reply, a Substitute 
Specification is provided. 

During a May 27, 2005 Telephone interview with Examiner Tabatabai, the Examiner 
noted some discrepancies between Applicants' specification and formal drawings. In 
particular, there was some concern over whether the specification matched the drawings. 

Applicants' representative noted to the Examiner that an April 4, 2002 Preliminary 
Amendment deleted reference to original drawing Figs. 17-19 and redacted the corresponding 
sections of the specification. Original Fig. 20 was renumbered as Fig. 17. Accordingly, the 
application now contains only Figs. 1-17. The Examiner appeared to be looking at the 
original specification, not the specification after entry of the Preliminary Amendment 

It was also discussed that original Figs. 14 and 16 were each broken down into Figs. 
14A, 14B and 16A, 16B, respectively, in the April 4, 2002 formal drawings. No new matter 
was added. The drawings were merely enlarged to make fit in two pages. These Figs, with A 
and B designations were not described in the specification. 

To place the application in condition for allowance, the Examiner during the May 27 
interview requested that Applicants provide a Substitute Specification that conforms the 
specification to the drawings. 

In reply, Applicants provide a Substitute Specification making the noted changes. In 
particular, the Substitute Specification makes the changes noted in the April 4, 2002 
Preliminary Amendment, which has been entered. The Substitute specification also changes 
reference to 'Tig. 14" to —Figs. 14A and 14B," as well as changes reference to "Fig. 16" to 
—Figs. 16A and 16B~. No new matter is added. 
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In view of the foregoing, it is respectfully submitted that this application is in 
condition for allowance. Favorable reconsideration and prompt allowance of claims 1, 3-9 
and 1 1-18 are earnestly solicited. 

Should the Examiner believe that anything further would be desirable in order to place 
this application in even better condition for allowance, the Examiner is invited to contact the 
undersigned at the telephone number set forth below. 




Registration No. 27,075 



Stephen P. Catlin 
Registration No. 36,101 



JAO:SPC/fpw 



Date: June 6, 2005 



Attachment: 

Substitute Specification 



DEPOSIT ACCOUNT USE 



Oliff & BERRUDGE, PLC 
P.O. Box 19928 
Alexandria, Virginia 22320 
Telephone: (703) 836-6400 



Charge any fee due to our 
Deposit Account No. 24-0037 



Please grant any extension 



AUTHORIZATION 



necessary for entry; 
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1 

WATERMARKED IMAGE GENERATOR AND METHOD OF EMBEDDING 
WATERMARKS INTO AN INPUT IMAGE 

Pt AnrraOT TND OF THE INVENTION 

5 1. Field of Invention 

[0001] This invention is related to embedding watermarks in input images to 

form watermarked output images. 
2. Description of Related Art 

10002] With the advent of digitization of images, digi tal image distribution 

10 and digital video availability, the "hiding" of information in digital images for 
purposes such as copyright protection has become a substantial issue for image 
publishers and authors. The process of imbedding information in a digital image is 
known as ••watermarking". Such watermarks must be secure, robust to intentional 
corruption and to compression processing, not unreasonably complex to embed and 

1 5 extract, and compatible and interoperable with conventional image processing 
systems. The watermark is generally invisible to. a viewer. However, in some 
applications, it is desirable to produce a visible watermark that can be removed by an 
authorized image decoder and that can not be removed by an unauthorized decoder. 
[0003] Many watermark algorithms have been proposed. However, only a 

20 few watermark algorithms have been designed for printed images. U.S. Patent 
5,734,752 to Knox discloses a method of using stochastic screens to create 
correlation-based digital watermarks. The concept of conjugate halftone screens for 
watermarking is proposed in U.S. Patent 5,790,703 to Wang. The methods disclosed 
in Knox and Wang might not be suitable for applications which require dynamically 

25 changing watermark information, because the watermarks in those methods can only 
be embedded during the screen design process for the halftone screen that will be used 
when printing a given image. U.S. Patent 5,946, 103 to Curry discloses a method that 
uses glyphs to digitally watermark a printed document. However, glyph encoding 
typically generates images with noticeable structures. This makes this method 

30 suitable onty for specific applications. 

SUMMARY OF THE INVENTION 
[0004) This inventi on provides systems and methods that embed watermarks 
in input images to produce output images having relatively high watermark resolution. 
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[0005J This invention separately provides systems and methods that embed 
watermarks in input images to produce output images having a relatively small 
amount of noticeable artifacts, 

[0006] This invention separately provides systems and methods that 
5 dynamically embed any type of watermark in an input image at run-time. 

[0007J In various exemplary embodiments of the systems and methods 
according to this invention, a watermark image generator includes a watermark data 
source that inputs watermark data to a watermark embedding device. The watermark 
embedding device halftones the input image to generate the output image made of 2x2 
10 binary patterns. The 2x2 binary patterns form the watermarks embedded in the output 
image. The watermark embedding device includes a tri-level error diffusion generator 
that performs tri-level error diffusion on the input image by halftoning the input image 
into black, white and 50% gray. A plurality of halftoning circuits successively replace 
each pixel of the halftoned input image with one of a plurality of the 2x2 binary 
15 patterns. The one of the plurality of 2x2 binary patterns corresponds to at least one 
respective bit of the watermark data when a pixel of the halftoned image data is 50% 
gray. 

[0008] In various exemplary embodiments of the watermarked image 
generator according to this invention, when a pixel of the halftoned input image data 
20 is 50% gray, the one of the 2x2 binary patterns forms one of a plurality of polarized 
patterns. The one of the plurality of polarized patterns is polarized according to the 
value of the respective bit of the watermark data. 

[0009] These and other features and advantages of this invention are 
described in, or are apparent from, the following detailed description of various 
25 exemplary embodiments of the systems and methods according to this invention. 

RR TPTF TT RSCRIPTIQN OF THE DRAWINGS 
[001 0] Various exemplary embodiments of this invention will be described 
in detail, with reference to the following figures, wherein: 

(001 1 1 Fig. 1 shows a first exemplary embodiment of the watermarked 
30 image generator according to this invention; 

[0012] Fig. 2 shows a first exemplary embodiment of a watermark 
embedding device according to this invention; 
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[0013] Fig. 3 shows the 2x2 binary patterns generated by the watermarked 
image generator according the first exemplary embodiment of this invention; 

[0014] Fig. 4 is a flowchart illustrating a first exemplary embodiment of the 
method of embedding watermarks in an input image according to this invention; 
5 [0015] Figs. 5 A-5D show patches of output images after being watermarked 

by the first exemplary embodiment of the watermarked image generator according to 
this invention; 

[0016] Fig. 6 is an error diffused image of a chapel incorporating a digital 
watermark generated according to this invention; 
10 [0017] Fig- 7A shows a portion of the digital version of the image shown in 

Fig. 6 after being watermarked by the first exemplary embodiment of the method of 
embedding watermarks in an input image according to this invention; 

[0018] Fig. 7B shows the same portion of the image shown in Fig- 6 that is 
shown in Fig. 7a after being printed and scanned; 
1 5 [001 9] Fig. 8A shows a portion of the digital version of the image shown in 

Fig. 6 after being watermarked by the first exemplary embodiment of the method of 
embedding watermarks in an input image according to this invention; 

[0020] Fig, 8B shows the same portion of the image shown in Fig. 6 that is 
shown in Fig, 8A after being printed and scanned; 
20 [0021] Fig. 9 shows one region of the image shown in Fig. 6 after being 

embedded with a bar code by the first exemplary embodiment of the method of 
embedding watermarks in an input image according to this invention; 

[0022] Fig. 10 shows another region of the image shown in Fig- 6 after being 
embedded with a bar code by the first exemplary embodiment of the method of 
25 embedding watermarks in an input image according to tbis invention; 

[0023] Fig. 1 1 shows the image shown in Fig. 6 after being embedded with a 
bar code by the first exemplary embodiment of the method of embedding watermarks 
in an input image according to this invention; 

[0024] Fig. 12 shows a second exemplary embodiment of the watermark 
30 embedding device according to this invention; 

[0025] Fig. 13 shows the 2x2 binary patterns generated by the watermarked 
image generator according to the second exemplary embodiment of this invention; 
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[0026] Fig. 11 is Fias. 14A and 14B show a flowchart illustrating a second 
exemplary embodiment of the method of embedding watermarks in an input image 
according to this invention; 

[0027] Fig. 15 shows a segmentation map embedding device that encodes a 
5 segmentation map before embedding the segmentation map into an image; 

[0028] ir. Figs. 16Aand 16B show a flowchart illustrating an 

exemplary embodiment of the method of embedding watermarks in an input image 
according to this invention;_ajjd 

[0029] — Fig. 17 shows a printed imago of on advertisement; 

10 [0030] — Fig. 1 8 ohows a s e gmentation map of th e image shown in Fig. 1 7; 

(00 31 ] Fig. 1 9 ohows the imago of Fig. 1 7 after being embedded witb its 

segmentation map by an exemplary embodiment of the method of embedding 

watermarks in on input imag e according to this invention; and 

[0032] — Fig. 20 is a flow ohart illustrating the embedding of bin aty- 

1 5 segm e ntation values into an input imag e u s ing the second embodime n t o f the method 
for omboddirig a wat e rmark into an image according to thio invention, 

[0032] Fig. 1 7 is a flow chart illustrating the embedding of binary 

segmentation values into an input image using th e second embodiment of the method 
for embedding a watermark into an im a^e according to this invention- 

20 DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS 

[0033] The various exemplary embodiments of the methods and apparatus 
according to this invention use a watermarking algorithm known as bipolar encoding 
through error diffusion. This algorithm employs polarized dot structures for data 
encoding. 

25 [0034] Fig. 1 shows one exemplary embodiment of a watermarked image 

generator 100 for embedding watermarks in an input image according to this 
invention. As shown in Figf 1 , an input image is input from an image data source 200 
over a link 210 to the watermarked image generator 100. It should be appreciated that 
the image data source 200 can be any device that stores and/or generates an electronic 

30 version of an image. 

[003S] Thus, the image can be a printed hard copy version of the image, and 
the image data source 200 can be a scanner that scans and outputs an electronic 
version of the image over the link 210 to the watermarked image generator 100. 
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Furthermore, the image data source 200 and the watermarked image generator 100 can 
be elements integrated into a photocopier. 

[0036] Similarly, the image data source 200 can be a server or other node on 
a local area network, a wide area network, an intranet, the Internet or any other 
5 distributed network. In this case, the image is already stored on the network in 
electronic form. 

[0037] Finally, the link 210 can be a wired or wireless link to the image 
input device 200. Thus, the image data source 200 and the link 210 can be any known 
or later-developed elements that are capable of supplying an electronic image to the 

10 watermarked image generator 100. 

[0038] As discussed above, the system and methods of this, invention work 
equally well on images that have not been transferred to hard copy. In this case, the 
image is already in digital form and the image is ready for processing by the 
watermarked image generator 100, 

15 [0039] The watermarked image generator 100 includes an input 

interface 1 10, a controller 120, a memory 130, one or more user input devices 140, 
and a watermark embedding device 150. The one or more user input devices 140, the 
image input device 200 and an image data sink 300 are connected to the watermark 
embedding device 100 via the input/output interface 110. 

20 [0040] In operation, grayscale image data is received from the image input 

device 200 via the link 210 and the input/output interface 1 1 0 of the watermarked 
image generator 100. The input/output interface 110 forwards the input image data 
received from the image input device 200, under the direction of the controller 120, to 
the watermark embedding device 150. A watermark is stored in the memory 1 30, or a 

25 particular watermark can be input by the user through the user input device 140 and/or 
the image data source 200. The watermark embedding device 1 50 halftones the input 
image data to output an image made up of 2x2 binary patterns. The 2x2 binary 
patterns form the watermarks that are embedded in the halftone output image. The 
halftone output image containing the visible watermark is then output, via 

30 input/output interface 110 and the link 310 to the image data sink 300. 

[0041] It should be appreciated that the image data sick 300 can be any 
device or combination of devices that outputs or displays the resulting halftone image 
data. Thus, the resulting image can be a printed version of the input image, and the 
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image data sink 300 can be a printer. Similarly, the image data sink 300 can be a 
monitor which is capable of displaying the electronic version of the resulting halftone 
image for viewing. Furthermore, the image data source 200, the watermarked image 
generator 100, and the image data sink 300 can be elements integrated into a single 
5 device, such as a digital photocopier. 

[0042] Similarly, the image data sink 300 can be a server or other node on a 
local area network, a wide area network, an intranet,, the Internet, or any other 
distributed network. In this case, the resulting halftone image is transferred and stored 
on the network in electronic form. Finally, the link 310 can be a wired or wireless 

10 link to the image data sink 300, to any other image output device or display device, or 
to the network. Thus, the image data sink 300 and the link 310 can be any known or 
later-developed systems and methods that are capable of receiving, and outputting or 
storing, the resulting electronic halftone image data generated by the watermarked 
image generator 100. < 

15 [0043] Fig. 2 shows a first exemplary embodiment of the watermark 

embedding device 150 according to this invention. As shown in Fig. 2, the watermark 
embedding device 150 includes a tri-level error diffusion generator 151, a decision 
circuit 152, and halftoning circuits 153, 155, 157 and 159. As will be described in 
greater detail, the watermark embedding device 150 halftones the input image to 

20 output an image made up of 2x2 binary patterns by using a two-stage halftoning 
process with error diffusion. 

[0044] The image input device 200 inputs the original grayscale or 
continuous tone image data to the tri-level error diffusion generator 151. The tri-level 
error diffusion generator 151 halftones the original continuous tone image data by tri- 

25 level error diffusion into three gray levels. This is performed at a reduced resolution. 
Specifically, each of the 2x2 pixels of the input image are grouped into one super 
pixel. The value of the super pixel is defined to be the average of the 2x2 pixels. The 
exror diffusion is then performed on the super pixels. The three gray levels to which 
the reduced-resolution continuous tone image is halftoned are black, white, and 50% 

30 gray. The process of error-diffusion, including tri-level and other multi-level error 
diffusion, is well known in the art. In general, any known or later-discovered error 
diffusion process is suitable for use with the systems and methods according to this 
invention. 
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[0045] After the input image is halftoned by the tri-level error diffusion 
generator 151, the halftoning circuits 153, 155, 157 and 159 replace each one of the 
pixels of the halftoned input image by a corresponding one of four 2x2 binary 
patterns. Which of the four 2X2 binary patterns is chosen to replace a particular pixel 
5 depends on whether the particular halftone pixel from the tri-level error diffusion 
generator is black, white, or 50% gray, and/or on the value of the binary watermark 
stored in or input to the watermarked image generator 150 for that particular pixel. 
The controller 120 sequentially feeds each bit of the binary watermark data stored in 
the memory 130 or input by the user through the one or more user input devices 140 
10 to the decision circuit 1 52. The source of the binary watermark data is shown 
generally in Fig. 2 as a binary data source 400. 

[0046] When a pixel of the halftoned input image is black, the decision 
circuit 152 is bypassed and the halftoning circuit 153 replaces the black pixel with an 
all black 2x2 binary pattern, as shown in part (a) of Pig. 3. When a pixel of the 
15 halftoned image is white, the decision circuit 152 is also bypassed and the halftoning 
circuit 159 replaces the white pixel with an all white 2x2 binary pattern, as shown in 
part (b) of Fig. 3. When a pixel is 50% gray and the next bit of the binary watermark 
data from the binary data source 400 is 1, the halftoning circuit 155 replaces the 50% 
gray pixel with a 2x2 binary pattern having black dots forming a pattern having a 45° 
20 polarized angle, as shown in part (c) of Fig. 3. When a pixel is 50% gray and the next 
bit of the binary watermark data from the binary data source 400 is 0, the halftoning 
circuit 1 57 replaces the 50% gray pixel with a 2x2 binary pattern having black dots 
forming a pattern having a 135° polarized angle, as shown in part (d) of Fig. 3. 

[0047] It should be appreciated that which 2x2 binary pattern chosen to 
25 replace the current pixel when the corresponding bit of the binary watermark data 

have a particular value is not limited to that shown in Fig. 4. For example, when a bit 
of the binary watermark data is 0, the corresponding pixel can be replaced with a 2x2 
binary pattern having a 45° polarized angle. 

[0048] The 2x2 binary patterns for the 50% gray level have two different 
30 orientations. However, they have the same gray appearance, which allows the 

watermark to remain hidden in the final image. The tri-level halftone output from any 
continuous-tone input, except solid black, solid white and 50% gray inputs, will be a 
spatial combination of the intermediate-level bipolar patterns and either the black or 
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the white patterns. Toggling the polarization between the two 2x2 binary patterns 
allows binary information to be arbitrarily embedded into any arbitrary image. 

(0049J Fig. 4 i$ a flowchart outlining a first exemplary embodiment of a 
method for embedding visible watermark into received images according to this 
5 invention. Beginning in step S100, operation continues to step S105 7 where 

continuous tone image data is input. Next, in step SI 10, the input continuous tone 
image data is halftoned by tri-level error diffusion. Operation continues to step SI 15. 

[0050] In step SI 15, a pixel is chosen from the halftoned image data. Next, 
in step SI 20, a determination is made whether the pixel of the tri-level output is 
10 black. If the pixel of the tri-level output is black, operation goes to step Si 25. 

Otherwise operation jumps to step S130. hi step SI 25, the black pixel is replaced 
with an all black 2x2 binary pattern. Operation then jumps to step Si 60. 

[0051] In step S 1 30, a determination is made whether the pixel of the tri- 
level output is white. If the pixel of the tri-level output is white, operation goes to 
15 step S135. Otherwise, operation jumps to step S140. In step S135 the white pixel is 
replaced with an all white 2x2 binary pattern. Operation then again jumps to step 
S160. 

[0052] In step Sl40 f a next bit of binary watermark data is input. Then, in 
step SI 45; a detennination is made, if the value of the next bit of the binary 

20 watermark data is 0. If the value of the next bit of the binary watermark data is 0 S 
operation proceeds to step S150. Otherwise, operation jumps to step S155. In step 
SI 50, the pixel of the tri-level output is replaced with a 2x2 binary pattern having a 
135° polarized angle. Operation then jumps to step SI 60. In contrast, in step SI 55, 
the pixel of the tri-level output is replaced with a 2x2 binary pattern having a 45° 

25 polarized angle. Operation then continues to step SI 60. 

[0053] In step S 160, a determination is made whether any pixels are left 
from the tri-level output. If there are pixels remaining, operation returns to step SI 15, 
where another pixel is chosen from the tri-level output. If there are no pixels 
remaining, operation continues to step SI 65, where operation of the method ends. 

30 [0054] Figs. 5 A-5D show patches of images after being watermarked by the 

methods and systems according to this invention. In Figs. 5A-5D, the 2x2 binary 
pattern shown in part (c) of Fig. 3 is chosen to replace a 50% gray pixel. 

[0055] Fig. 5 A shows the output of the watermarking embedding device 
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according to this invention when the constant input represents a dark shadow region, 
or 0<u«128 f where u is (he gray level of the input based on a 256 level scale. Li this 
case, the white dots form patterns having a 45° polarized angle. 

[00S6J Fig. 5D shows the output of the watermarking embedding device 
5 according to this invention when the constant input represents a highlight region, or 
128«u<255- In this case, the black dots form a pattern having a 135° polarized 
angle. 

[0057] When the gray level u of the constant input is exactly equal to 128, a 
perfect checkerboard pattern will form, and no information can be embedded. 

1 0 However, when the gray scale level u of the constant input is close to, but not equal 
to, 128, information is embedded in the input image at a larger scale. For example, 
Fig. 5D shows the output of the watermarking embedding device according to this 
invention when the constant input represents a dark mid-tone region, or 0«u<128. In 
this case, large size black dots are formed having a 45° polarized angle. Fig. 5C 

1 5 shows the output of the watermarking embedding device according to this invention 
when the constant input represents a light mid-tone region, or 128<u«255. In this 
case, large size white dots are fonned having a 135° polarised angle. 

(0058] The following illustrative examples are made in reference to the 
image shown in Fig. 6. Fig. 6 shows an error diffused image of a chapel printed at 

20 600 dpi. 

EXAMPLE 1 : 

[0059] The watermark embedding method according to this invention is 
coded in C- All test images are halftoned and printed at 600 dpi, and then scanned at 
600 spots per inch (spi). First, the image shown in Fig. 6 is halftoned by the method 

25 according to this invention using only the 2x2 pattern shown in part (c) of Fig. 3 to 
replace the intermediate level. Fig. 7A shows a portion of the digital version of the 
image shown in Fig. 6 after being watermarked by the method according to this 
invention- Fig. 7B shows the same portion of the image shown in Fig. 7A after being 
printed and scanned. The four distinct polarized structures shown in Figs. 5A-D are 

30 visible in the images of Figs. 7 A and 7B. 
EXAMPLE 2: 

[0060] The same as Example 1, except only the 2x2 pattern shown in part 
(d) of Fig. 3 is used to replace the intermediate level. Fig. 8 A shows a portion of the 
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digital version of the image shown in Fig. 6 after being watermarked by the method 
according to this invention. Fig. 8B shows the same portion of the image shown in 
Fig. 8A after being printed and scanned. Figs. 8A and 8B are identical to Figs. 7A 
and 7B, except for the different polarization of the watermarks. 
5 EXAMPLE 3: 

[0061] The watermarking method according to this invention is used to 
embed a bar code in the image shown in Fig. 6. One exemplary pattern for the one- 
dimeasional bar code is {1,0,1,1, 1,0,0,1,0,1}- Each bit of the bar code is encoded 
using two scaolines, and the bar code is repeated throughout the image. Two regions 
10 of the watermarked image are shown in Figs. 9 and 10. The dot structures having 
different orientations are visible in Figs. 9 and 10. Fig. 1 1 shows the entire image 
shown in Fig. 6 watermarked with the bar code. A comparison of the image shown in 
Fig. 1 1 with the image shown in Fig. 6 illustrates that the watermarking method 
according to this invention does not cause significant degradation of image quality. 
1 5 [0062] Fig. 12 shows a second embodiment of a watermark embedding 

device 500 according to this invention. As shown in Fig. 12, the watermark 
embedding device 500 includes a tri-level error diffusion generator 510, a decision 
circuit 520, and halftoning circuits 530, 540, 550, 560, 570 and 580. As in the first 
embodiment shown in Fig. 2, the tri-level error diffusion generator 510 halftones a 
20 reduced resolution version of the original input continuous tone image data by tri- 
level error diffusion into three gray levels. The three gray levels to which the original 
image is halftoned are black, white, and 50% gray. 

[0063] After the input image is halftoned by the tri-level error diffusion 
generator 510, the halftoning circuits 530, 540, 550, 560, 570 and 580 replace each 
25 one of the pixels of the halftoned input image by a corresponding one of six 2x2 

binary patterns. Which of the six 2x2 binary patterns chosen to replace a particular 
pixel depends on whether the particular halftone pixel is black, white, or 50% gray, 
and/or on the value of the binary watermark stored in or input to the watermarked 
image generator 500 for that particular pixel, bi this embodiment, the controller 120 
30 sequentially feeds two bits, rather than a single bit as in the first embodiment, of the 
binary watermark data stored in the memory 1 30 or input by the user through the user 
input device 140 to the decision circuit 520, The source of the binary watermark data 
is shown generally in Fig. 12 as the binary data source 600. 
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[0064 J When a pixel of the halftoned input image is black, the decision 
circuit 520 is bypassed and the halftoning circuit 530 replaces the black pixel with an 
all black 2x2 binary pattern, as shown in part (a) of Fig. 13. When a pixel of the 
halftoned image is white, the decision circuit 520 is again bypassed and the halftoning 
5 circuit 580 replaces the white pixel with an all white 2x2 binary pattern, as shown in 
part(b) of Fig. 13, 

[0065] When a pixel is 50% gray and the bits of the binary watermark data 
are "00," the halftoning circuit 540 replaces the 50% gray pixel with a 2x2 binary 
pattern having black dots forming a pattern having a 0° polarized angle, as shown in 
10 part (c) of Fig. 13. When a pixel is 50% gray and the bits of the binary watermark 

data are "01," the halftoning circuit 550 replaces the 50% gray pixel with a 2x2 binary 
pattern having black dots forming a pattern having a 45° polarized angle, as shown in 
parted) of Fig. 13. 

(0066J When a pixel is 50% gray and the bits of the binary wateimark data 
15 are "10," the halftoning circuit 560 replaces the 50% gray pixel with a 2x2 binary 

pattern having black dots forming a pattern having a 90° polarized angle, as shown in 
part (e) of Fig. 13. When a pixel is 50% gray and the bits of the binary watermark 
data are "1 1," the halftoning circuit 570 replaces the 50% gray pixel with a 2x2 binary 
pattern having black dots forming a pattern having a 1 35° polarized angle, as shown in 

20 part (f) of Fig. 13. 

[0067] According to the second embodiment, the 2x2 binary patterns for the 
50% gray level have four different orientations. However, they have the same gray 
appearance, which allows the watermark to remain hidden in the final image. The tri- 
level halftone image data output from any continuous tone input image, except solid 

25 black, solid white and 50% gray inputs, will be a spatial combination of the 

intermediate-level bipolar patterns and either the black or the white. Toggling the 
polarization between the four 2x2 binary patterns allows binary information to be 
arbitrarily embedded into an arbitrary image. 

[0068] Fig. 14 ia Fias. 14A and 14B show a flowchart outlining a second 

30 exemplary embodiment of a method for embedding visible watermark into received 
images according to this invention. Beginning in step S200, operation continues to 
step S205, where continuous image data is input. Next, in step S210, the input 
continuous tone image data is halftoned by tri-level error diffusion, hi step S215, a 
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pixel is chosen from the halftoned image data. Next, in step S220, a determination is 
made whether the pixel of the tri-level output is black. If the pixel of the tri-level 
output is black, operation goes to step S225. Otherwise, operation jumps to step 
S230. In step S225, the black pixel is replaced with an all black 2x2 binary pattern, 
5 Operation then jumps to step S280. 

[0069] In step S230, a determination is made whether the pixel of the tri- 
level output is white. If the pixel of the tri-level output is white, operation goes to 
step S235. Otherwise, operation jumps to step S240. In step 8235, the white pixel is 
replaced with an all white 2X2 binary pattern. Operation then jumps to step S280. 
10 [0070] In step S240, the next two bits of binary watermark data are input. 

Then, in step S245, a determination is made if the value of the next two bits of the 
binary watermark data are 00. If the value of the next two bits of the binary 
watermark data are 00, operation proceeds to step S250- Otherwise, operation jumps 
to step S255. In step S250, the pixel of the tri-level output is replaced with a 2x2 
15 binary pattern having a 0° polarized angle. Operation then jumps to step S280. 

[0071 ] In step S255, a determination is made if the value of the next two bits 
of the binary watermark data are 0 1 . If the value of the next tw o bits of the binary 
watermark data are 01, operation goes to step S260- Otherwise, operation jumps to 
step S265. In step S260, the pixel of the tri-level output is replaced with a 2x2 binary 
20 pattern having a 45° polarized angle. Operation then jumps to step S280. 

[0072] In step S265, a determination is made if the value of the next two bits 
of the binary watermark data are 10. If the next two bits of the binary watermark data 
are 10, operation goes to step S270. Otherwise, operation jumps to step S275. In step 
S270, the pixel of the tri-level output is replaced with a 2x2 binary pattern having a 
25 90° polarized angle. In contrast, in step S275, the pixel of the tri-level output is 
replaced with a 2x2 binary pattern having a 1 35° polarized angle. Operation then 
continues to step S280. 

[0073J In step S280, a determination is made whether any pixels are left 
from the tri-level output. If there are pixels remaining, operation returns to step S215, 
30 where another pixel is chosen from the tri-level output. If there are no pixels 

remaining, operation continues to step S285, where operation of the method ends. 

[0074} It should be appreciated that which 2x2 binary pattern chosen to 
replace the current pixel when the corresponding two bits of the binary watermark 
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data have particular values is not limited to that shown in F*gr4 4Pigs. 14Aand 14B . 
For example, when the bits of the binary watermark data are 00, the corresponding 
pixel can be replaced with a 2x2 binary pattern having a 45° polarized angle. 

[0075] The various exemplary embodiments of the systems and methods 
5 according to this invention can be used to embed many types of information in an 

original image. For example, the methods and systems according to this invention can 
be used to embed a segmentation map in a printed document image. Document 
images can be segmented into regions that have distinct properties, such as text, 
pictures and graphics. A segmentation map is an indexed image, where the index of a 

10 pixel is the class label of a corresponding pixel or a corresponding group of pixels in 
the original image. Because regions having different characteristics need to be 
processed and rendered differently, image segmentation is crucial for generating or re- 
producing high quality document images. 

[0076] Conventionally, the segmentation map of a document image is often 

1 5 only available before the image is printed. However, it is discarded when the image is 
rendered. If the printed image is digitally scanned* the segmentation map needs to be 
generated again, since it is difficult to recover from the printed document. Thus, it is 
desirable to embed the segmentation map in the printed document image for fast and 
accurate retrieval. 

20 [0077] A segmentation map can be embedded into an image either with 

encoding or without encoding. When a segmentation map is not encoded, the 
segmentation map is directly embedded into an image, hi this case, the segmentation 
map is directly correlated with the image content. Thus, even if the printed image is 
altered, such as cropped, or rotated, the segmentation map can still be recovered. 

25 Embedding the segmentation map without encoding also improves robustness to 

detection errors. That is, in this case, the effect of a wrong detection can be locally 
contained. 

[0078J On the other hand, by embedding with encoding, the amount of data 
that needs to be embedded can be drastically reduced. Fig. 15 shows a segmentation 
30 map embedding device that encodes a segmentation map before embedding the 
segmentation map into an image. The segmentation map embedding device 600 
includes an image data source 610, a segmentation map generator 620, an encoder 
630, and a watermarked image generator 640. The image data source 610 inputs a 
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digital image to the segmentation map generator 620. The segmentation map 
generator 620 generates a segmentation map of the image input by the original image 
source 610. The segmentation map is then input to the encoder 630. The encoder 630 
encodes the segmentation map. The encoder 630 can encode the segmentation map by 
5 any suitable encoding process. For example, if all regions in a document image are 

assumed to be rectangular in shape, then each region can be coded using four numbers 
that specify the location and the dimension of a rectangular, followed by a class label. 
Generally, a segmentation map can be first compressed using either run-length coding, 
or predictive coding, followed by entropy coding. 

1 0 [0079] After the segmentation map is encoded by the encoder 630, the 

encoded segmentation map is treated as a binary data source. The binary data from 
the encoded segmentation map is input to the watermarked image generator 640. The 
watermarked image generator 640 embeds the binary data from the encoded 
segmentation map in the image as watermarks using the various exemplary 

15 embodiments of the systems and methods according to this invention. 

[0080] Fiq. 16 ifl F igS- 16A and 16B show a flowchart outlining the 
embedding of binary segmentation values into an input image using the second 
embodiment of the method for embedding a watermark into an image according to 
this invention. Beginning in step S300, operation continues to step S305 where 

20 continuous tone image data is input. Next, in step S31 0, the input continuous tone 
image data is halftoned by tri- level error diffiision. In step S3 1 5, a pixel is chosen 
from the halftoned image data. Next in step S320, a determination is made whether 
the pixel of the tri-level output is black. If the pixel of the tri-level output is black, 
Operation goes to step S325. Otherwise, operation goes to step S330. In step S325, 
25 the black pixel is replaced with an all black 2x2 binary pattern. Operation then jumps 
to step S3 80. 

[0081 ] In step S330, a determination is made whether the pixel of the tri- 
level output is white. If the pixel of the tri-level output is white* operation goes to 
step S335. Otherwise, operation goes to step S330. In step S325, the white pixel is 
30 replaced with an all white 2x2 binary pattern. 

[0082] In step S340, the next two bits of the binary segmentation map data 
are input. 
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[0083] Operation then continues to step S345. In step S345, a determination 
is made whether the values of the next two bits of the binary segmentation map data 
are 00. If the values of the next two bits of the binary segmentation map data are 00, 
then operation goes to step S210- Otherwise, operation goes to step S355. Instep 
5 S3 50, the current pixel is replaced with a 2x2 binary pattern having a 0° polarized 
angle. Operation then jumps to step S380. 

[0084] In step S355, a determination is made whether the values of the next 
two bits of the binary segmentation map data are 01 . If the values of the next two bits 
of the binary segmentation map data axe 01, then operation goes to step S360. 
10 Otherwise, operation goes to step S365. In step S360, the current pixel is replaced 

with a 2x2 binary pattern having a 45° polarized angle. Operation then jumps to step 
S380. 

[0085] In step S365, a determination is made whether the values of the next 
two bits of the binary segmentation map data are 1 0. If the values of the next two bits 

15 of the binary segmentation map data are 1 0, then operation goes to step S370- 

Otherwise, operation goes to step S375. In step S370, the current pixel is replaced 
with a 2x2 binary pattern having a 90° polarized angle. In contrast, in step S3 75, the 
current pixel is replaced with a 2x2 binary pattern having a 135 ° polarized angle. 
[0086] In step S3 80 5 a determination is made whether any pixels are left 

20 from the tri-level image. If there are pixels remaining, operation returns to step S3 15, 
where another pixel is chosen from the tri-level output. If there are no pixels 
remaining, operation continues to step S385, where operation of the method ends. 

[0087] It should be appreciated that which 2x2 binary pattern chosen to 
replace the current pixel when the corresponding two bits of the binary segmentation 

25 map data have particular values is not limited to that shown in Fig. 16 F igs. 16A and 
16B . For example, if the next two bits of the binary segmentation map data are 00, 
the current pixel can be replaced with a 2x2 binary pattern having a 135° polarized 
angle. 

[0088] — The following illustrativ e examples ar e made in reference to tho - 

30 imag e shown in Fig. 17. 
EXAMPLE 4: 

[0089] — Tho third embodiment of tho watermark emb e dding method 

according to this - in ve ntion ig cod e d in C and tested. All toot images-are halfton e d and 
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printed at 600 dots per ino fa (dpi), then scanned at - 6QQ dpi. Each 2x2 bl o ok of pixels 
is manually classifi e d into thr e e class e s: pictorial regiofts-text and graphico on solid 
bac k ground, and text and graphics on halftono background. The resulting 
segmentation map is shown in Fig. 1 8 , In Fig.- t-fi, tho black regions indioato pictur e s, 
5 th e gray regions indicate text on halfton e background, and th o whit e r e gions indicate 
t e xt on solid ba c kground. The s e gm e ntation map is directly embedded in tho origirat l- 
kna go shown in Fig. i 7, without encoding. The resulting waterma rke d image is 
shown - is r- Fig. 1 9. The throo s e gmentation classes are repr e sented by tho 45°, 90° and 
135° 2X2 p olarizod patt e rns shown in parts (d) (f) of Fig. 13, r e sp e ctively. 

10 100891 The third embodiment of the watermark embedding method 

according to this invention is coded in C and tested All test images are halftoned and 
printed at 600 dots per inch fdpi>, then scanned at 600 dpi. Each 2x2 block of pixels 
is manually classi fied into three classes: pictorial regions,, text and graphics on solid 
background, and text and graphics on halftone background. The black regions 
15 indicate pictures, the gray regions indicate text on halftone background, and the white 
regions indicate text on solid background. The segmentation map is directly 
embedded in the original image, without encoding. The three segmentation classes 
are represented bv the 45°. 90° and 133° 2X2 polarized patterns shown in parts (dWft 
of Fig. 13. respectively^ 

20 {0090] — A comparison of tho watermark e d image sk e- wn in Fig. 19 with tho - 

corresponding original imago shown in fig. 17 illustrat e s that tho watermarking 
method according to this invention does not oaufio Gignificatit d e gra d ation of imag e 
quality. 

[0091] The retrieval of watermarks embedded by the various exemplary 
25 embodiments of the systems and methods according to this invention requires only the 
detection of the polarizations of the dot structures on the watermarked image. Thus, 
retrieval is relatively easy and robust. One exemplary embodiment of a method of 
retrieving watermarks embedded by the various exemplary embodiments of the 
systems and methods according to this invention includes scanning the printed image 
30 with a scanner having a resolution at least equal to the resolution of the embedded 
watermark. The watermarks in the digitized image can be detected using pattern 
matching methods that are well known in the art. 
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10002 ] — Fig. 20 is a flowchart outlining one exemplary embodiment of a 

mothod of retrieving watermarks embedded by th e various e xemplary embodim e nts of 
the-syste gas - and methods according to this inv e ntion. B e ginning in - Dtcp S10Q, 
operation eo ntinu e s to step S 4 05, wherc4b e-s vat e rmark e d image data is input. Noxl 3 
5 in step 10, a 2x2 pix e l pattern is chosen from the wate r marked image data. Next, in 
' step S4 15 , a determination is mode wh e ther the 2x2 pixel pattern io oil whit e or all 
black. If the 2x2 pb i dl pattern is a]] white or all black, operation goes to st e p 4 55, 
Oth e rwis e , operation goes to st e p S420. 

f 00921 Fia. 17 is a flowchart outlining one exemplary embodiment of a 

10 method of retrieving watermarks embedded by the various exemplary embodiments of 
the systems and methods according to this invention, BeginningJfl step S400. 
operation continues to step S405. where the watermarked image data is input. Next, 
in step S41 0, a 2x2 pixel pattern is chosen from the watermarked image data. Next, in 
step S41 5, a determination is made whether the 2x2 pixel pattern is all white or all 
15 black. If the 2x2 pixel pattern is all white or all black, operation goes to step 455. 
Otherwise, operation goes, to step S420. 

[0093] In step S420, a determination is made whether the 2x2 binary pattern 
has a 0° polarized angle. If the 2x2 binary pattern has a 0° polarized angle, operation 
goes to step S425. Otherwise, operation goes to step S430. In step S425, the values 
20 of the next two bits of the watermark data are determined to be 00. Operation then 
jumps to step S455. 

[0094] In step S430, a determination is made whether the 2x2 binary pattern 
has a 45° polarized angle. If the 2x2 binary pattern has a 45° polarized angle, 
operation goes to step S435. Otherwise, operation goes to step S440. In step S435, 
25 the values of the next two bits of the watermark data are determined to be 01. 
Operation then jumps to step S455. 

[0095] In step S440, a determination is made whether the 2x2 binary pattern 
has a 90° polarized angle. If the 2x2 binary pattern has a 90° polarized angle, 
operation goes to step S445. Otherwise, operation goes to step S450. In step S445, 
30 the values of the next two bits of the watermark data are determined to be 10. In 
contrast, in step S450, the values of the next two bits of the watermark data are 
determined to be 1 1. Operation then continues to step S455. 
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[0096| In step S455, a determination is made whether any pixels are left 
from the watermarked image data. If there are pixels remaining, operation returns to 
step S410, where another 2x2 binary pattern is chosen from the watermarked image 
data. If there are no pixels remaining, operation continues to step S460, where 
5 operation of the method ends. 

[0097] — It should bo appr e ciated that which values for th e n e xt two bits of the 

watermark chosen to correspond to th e 2x2 binary pattern of the binary watermarked 
imag e data is not - fegut e d to thos e shown in Pi£. 20. For e xampl e , if th e 2x2 bin aiy- 
patt e rn has a 90° polorizod - onglo, the values of th e n e xt two te E s - of - tho watermark data- 

10 con b o d o t e rmin e d to b e 1 1. 

[00971 It should be appreciated thai which values for the next two bits of the 

watermark chosen to correspond to the 2x2 binary pattern of the binary watermarked 
image data is not limited to those shown in Fig. 17. For example, if the 2x2 binary 
pattern has a 90° polarized angle, the values of the next two bits of the watermark data 

15 can be determined to be 11. 

[0098] While this invention has been described in conjunction with the 
specific exemplary embodiments outlined above, it is evident that many alternatives, 
modifications and variations will be apparent to those skilled in the art. Accordingly, 
the exemplary embodiments of the invention, as set forth above, are intended to be 

20 illustrative, not limiting. Various changes may be made without departing from the 
spirit and scope of the invention. 
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